In the modern world of e-commerce, social media, and cloud computing the financial stakes are enormous when a company's website crashes. For example, website downtime for a company like Twitter® has been estimated at roughly $25 million dollars per minute. Therefore it is imperative that a business enterprise adequately test their websites and web applications prior to initial launch—not only to avoid catastrophic failures, but also to ensure the best user experience possible.
Real-time, web-based, application performance testing, also known as load testing, utilizes the global supply of regional cloud providers to run tests on a website or web application by simulating the browsing activities of hundreds of thousands—or even millions—of concurrent virtual users. Cloud-based load testing makes it possible for a business enterprise to simulate huge numbers of users on a website within a matter of minutes. Performance or load testing thus provides a company with the ability to determine a system's behavior under both normal and anticipated peak load conditions. Further, load testing can be used to identify the maximum operating capacity of an application and also determine which components are causing performance problems or degradations.
In one known approach, an array of load servers is deployed in the cloud to generate a test load on a target website. By way of background, U.S. Pat. No. 8,341,462, which is herein incorporated by reference, describes a computerized method that creates a grid comprising a collection of interconnected load servers and result servers that may then be used to run a load test on a target website or web application. The load servers play a predefined test composition, which generates a load on a target (customer) website or web application. The load generated by the load servers typically consists of a trapezoidal-shaped ramp profile, with a linear ramp-up of a predetermined number of virtual users (e.g., 100,000) followed by a flat, plateau time duration where the number of virtual users remains constant, and then a gradual ramp-down period. While the test composition is running, the load servers also function to report back results of the load test and statistics in real-time. Analytic or result servers are deployed and utilized primarily to collect the real-time test results from the load servers, aggregate those results, and store them in a database. By way of further background, U.S. Pat. No. 7,844,036, which is herein incorporated by reference, discloses a graphical user interface (GUI) that allows a user to compose a test scenario by dragging and dropping various message clips into tracks to generate complex parallel message streams for load testing purposes.
Enterprises are interested in changing the virtual user distributions on-the-fly while the load test is running. That is, they would like to change the number of virtual users, increasing or decreasing distributed virtual user traffic loads while test compositions are running. Traditional load testing systems and tools have lacked the ability to make changes and adjustments to virtual user distributions and profiles without stopping or rescheduling the test.